Lenslet Image Coding With SAIs Synthesis via 3D CNNs-Based Reinforcement Learning With a Rate Reward

The deep learning-based coding schemes for lenslet images combine standards and view synthesis through Deep Learning (DL) models, where the compression efficiency is heavily influenced by structure quality of synthesized views. To exploit inter-view redundancy among Sub-Aperture Images (SAIs), this paper proposes a hybrid closed-loop system that uses novel based on checkerboard interleaving at frame level. frame-wise method partitions an Original SAIs’ Set (OSS) into two mutually exclusive subsets, each consisting alternating rows columns SAIs. We utilize video standard Versatile Video Coding (VVC) to encode one subset while proposing rate constraint-reinforced 3D Convolutional Neural Networks (CNNs) predict other subset, referred as complement subset. CNNs newly designed with gradient loss reinforced cost improve image bit saving simultaneously. Experimental results light field dataset demonstrate proposed outperforms both HEVC_LDP previous state-of-the-art (SOTA), achieving average BD-Bitrate savings 41.58% 23.31%, respectively.